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IMPROVING TEACHER EDUCATION 
THROUGH PROGRAM EVALUATION 



Teacher education is a major industry in North America. Teachers 
comprise a significant portion of the work force and their impact 
on society cannot be overestimatec Recently there have been 
major criticisms of the educational system as a whole, and of 
teachers who are incompetent, and concerns about children who 
leave school without the basic skills. As a result of these 
criticisms teacher evaluation is receiving a high profile, 
particularly in Alberta where the Minister of Education has 
legislated mandatory evaluations of all teachers , 

But what of the p'^ograms that prepared these teachers in the 
first place? Ho fective are they? One might have expected, 
given the enormous investment of dollars in preparing teachers, 
that teacher education programs would be evaluated regularly. 
This is simply not the case. Evaluation studies of teacher 
education programs are conspicuously absent in the literature on 
teaching and teacher education. This is not to say that there 
has been no research on teacher education (the literature 
contains hundreds of studies on various aspects and components of 
teacher education) , but comprehensive evaluation studies designed 
to provide program planners and decision-makers with useful 
information are few and far between. 

It is the purpose of this paper to describe an ongoing 
comprehensive model of program evaluation which has as its major 
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goal the improvement of teacher education. The project is 
different from others you will be hearing about (in fact, 
''project" is a misnomer); it has no defined beginning or end; it 
is clearly formative; it is fluid and multi-faceted and is 
constantly changing and developing. I think of it as being 
something like a spider web; when you touch oae thread everything 
moves and changes; the bonds are delicate and easily broken, but 
they are there, and however tentatively, they do hold everything 
together • 

I plan to describe this project in four parts: 

1. the process - how and why the project developed or is 
developing as it is 

2. the conceptual frajnework, or the model within which 
the evaluation occurs 

3. implementation - specific evaluation projects within 
the conceptual framework, and 

4. the administrative framework for facilitating utiliza- 
tion of the evaluation results. 

THE PROCESS 

The University of Lethbridge teacher ^*<^ucation program has from 
its inception considered evaluation to be an integral part of its 
philosophy. Within a very few years of its beginning a major 
project known as QAULTEP (the Qualitative Analysis of the 
University of Lethbridge Teacher Education Prograin) had been 
established. (For a complete description of QAULTEP see Dravland 
and Greene, 1070 and Greene, 1G81, Chapter 3). By 1982 the 
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QAIJLTEP data base contained over 400 bits of information on 
almost every student (N>1000) who had been through the program. 
Some 30 reports had been written about various aspects of the 
program and the system was considered to be one oi few 
comprehensive programs for evaluating teacher education 
(McCutcheon, 1979; Peck, 1981, Note 1)- A report written on the 
evaluation program describes a large number of projects conducted 
under its auspices (Greene, 1979), However, for a variety of 
reasons, primarily declining resources coupled with a perceived 
lack of impact of the QAULTEP studies, the project was halted in 
1982. A Program Evaluation Committee was created and was charged 
with reviewing the model and recommending a new or revised system 
for program evaluation. 

University committees are known to work with great speed and 
efficiency; thus by 1984 (two years 7ater) the committee had 
presented The Faculty of Educat^ion Council with an administrative 
fram«;work for evaluating the program (to be discussed later) , and 
a tentative outline of a conceptual framework for program 
evaluati The committee requested interested members to 

participate in the further development of the framework; the 
resulting program evaluation "group" consisted of almost 
one-third of the Faculty, indicating that this endeavor received 
considerable support and commitment. During the development 
process interested persons from programs in Saskatchewan and 
British Columbia participated in discussions and there was 
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consultation with and input from four visiting scholars. 
Finally, a Tri-Universities committee consisting of a 
representative from each of the three Alberta universities was 
created and has since begun to further develop and implement 
aspects of the model. (See page 16 for a listing of participants 
and consultants) . 

THE CONCEPTUAL FRAMEWORK 

Time does not permit me to explain the rationale behind the 
framework which ultimately resulted from the group's delibera- 
tions. Major influences on the group's tl inking were previous 
experiences with QAULTEPi developments and criticisms of previous 
evaluations (see for example Greene, 1084, pp. 12-28) and 
recommendations of educational program evaluation leaders such as 
Turner (1975), Schalock (1980), Cooper (1980) and others. We 
were guided by a ntimber of principles that we believed our 
framework should incorporate, namely: 

1. it should be comprehensive, longitudinal and 
multi-faceted 

2. it should incorporate a wide variety of research and 
evaluation designs and methodologies 

3. it should be nrimarily internal; that is most of the 
evaluation should be done by members of the Faculty, 
but it should also allow for external evaluations 

4. it should allow for and facilitate collaborative and 
cooperative research with other institutions 

5. it should provide useful information for those 
responsible for making decisions about teacher 
education programs, and finally. 
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6. it should be possible , given the limited resourses 
available . 

In addition to these guiding principles with respect to process , 
we believed that the evaluation of teacher education programs 
should include the evaluation of all of the following aspects: 

1. students ~ selection and development 

2. graduates - placement , competence , and effectiveness 

3. faculty ~ expertise , and effectiveness 

4. programs - courses, modules, pracbicums, integraticxi 
of theory and practice, and 

5. resources and facilities. 

The resulting conceptual framework is shown in Figure 1. The 
model suggested for guiding the evaluation of teacher education 
programs is based on Turner's 1Q75 schema ^ but collapsed for our 
purposes into three categories: selection, program and work 
success. The specific items within each category are those 
suggested by experience and by recent research on teacher 
education as being the most important and productive. 
(References are available for each of the items, but have not 
been included in this paper; however, see for example Cooper 
(1980); Cruickshank (1984); Loadman, 1984). 

The framework is not intended to be exhaustive or exclusive « Its 
major function is to provide a vehicle for coimnunication among 
western Canadian teacher education institutions actively engaged 
or interested in evaluating their programs. It provides a focus 
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for research discussions, a framework for designing collaborative 
projects I a basis for collecting and sharing common data, and an 
opportunity for sharing research findings. It is flexible enough 
to accommodate any teacher education program and to be 
context-specif ic; ' that is, it can be adapted to fit the goals or 
philosophy of a specific teacher education program, but it also 
allows for the generalizability of research findings. It 
promotes longitudinal research, allows for individual, group and 
collaborative research and can be fit into an administrative 
schema for decision-making (see the University of Lethbridge 
schema in Figure 2) . 

Nor is the model intended to suggest that research would progress 
in a linear fashion from one category to the next. Rather, there 
are a number of possibilities: for example, data collected on 
students at entrance could be collected at various stages in the 
program to determine how students are charging and at what points 
in the program those changes are occurring; research on "good^ or 
effective teaching, and school-based research on student learning 
could be used to guide program development, and there are endless 
other possibilities. 

IMPLEMENTATION OF FRAMEWORK 

It is obvious that no one, or even ten, evaluation projects can 
assess definitively the effectiveness of a teacher education 
program. Therefore the process being implemented at the 
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University of Lethbridge is to conduct specific projects within 
the overall frauneworki the idea being that each will contribute 
to improving teacher education i and that taken together their 
impact will be greater than simply a sum of the individual 
studies c This framework was developed only one year ago axid 
there are presently at least five new evaluation projects 
underway. I'd like to describe some of these very briefly. The 
first three of these studies are being conducted cooperatively in 
all three Alberta teacher education institutions. 

I ~ Selection and Development of Teacher Education Candidates 

The purpose of this study is to assess and compare the 
characteristics and qualities of teacher education candidates 
admitted to the three Alberta teacher education institutions i and 
to assess the development of those qualities at various stsiges 
during the three different programs. This study fits primarily 
within the first two categories of the framework (Figure 1) but 
has implications well beyond the program itself. 

The project will examine specifically the criteria used to select 
teacher education candidates > qualities and characteristics of 
students admitted to teacher education (e.g., life experiences, 
academic and basic skills , cognitive skills and personal 
qualities) , developments in students with respect to these 
qualities y and differences within and among the Alberta programs 
relative to candidate characteristics and development. 
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The study is being conducted in the three Alberta teacher 
education institutions. A sample of 30 teacher education 
candidates at each institution has been selected on admission to 
the programs and is being administered a packet of instruments 
selected according to their technical quality, credibility, and 
usefulness as assessed from the teaching literature. The 
students will be tested again at intervals during the program and 
at exit from the program. Data analysis will be both descriptive 
and statistical, and both cross-institutional and longitudinal. 
The result will be a profile or profiles of Candidates on 
admission to the three institutions, and at various points during 
the various programs. Ultimately these profiles will be compared 
with those of effective teachers. 

II - Teachers' Perceptions of Their Education Programs 

This study, too, is a joint project of the three institutions but 
it has been sponsored and funded by Alberta Education and by the 
Alberta Teachers' Association. The purpose of this study is to 
provide evaluative information to teacher education programs by 
assessing graduates' perceptions of the effectiveness of their 
preparation for teaching. A survey instrument and an interview 
instrument based on the teaching effectiveness literature are 
being used to determine 1) how important teachers believe these 
skills, competencies and knowledge to be for teachers, 2) the 
potential contribution of preservice programs to the development 
of these skills and knowledge, and 3) the actual contribution of 
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their preservice programs to the development of teaching 
knowledge and skills* Comparisons are being made sjoong first , 
third and fifth year teachers , and within and among the three 
teacher education programs. 

Ill - The Evaluat ion of the Preservice Competence of Alberta 
Teachers 

The third study being conducted by this group, again within the 
overall framework of program evaluation in teacher education , is 
a study of the competence of new teacher education graduates • 
This study, too, is being funded by Alberta Education. The 
purpose of this study is to develop and test empirically based 
methods and procedures for evaluating preservice teacher 
connpetencies . We have expanded the traditional definition of 
competence to include not only behavioral and perfoi*mance skills, 
but also pedagogical and content knowledge and personal 
qualities* In addition to simply developing the procedures we 
intend to administer these to a sample of graduates at each 
university ir order to develop profiles of the competence (as we 
have defined it) of newly graduated Alberta teachers. 

Specific Tasks > This project has been written in terms of 
tasks rather than questions. These tasks include the following: 

1. To perform a search of the literature for the purpose 
of investigating which skills, behaviors, and 
qualities are generally associated with preservice 
competence in graduate teachers. 

2. To determine from the research literature what methods 
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and procedures are used for the assessment of th^ 
skills in (1) above. 

3. To determine what, if any, screening for preservice 
competence occurs at Faculties of Education in Canada. 

4. To cooipile a tentative battery of empirically based 
methods for administration to a sample of graduating 
B.Ed, candidates in Alberta. 

5. To administer the battery to a sample at each 
university . 

6. To perform statistical analysis on the results of the 
administration for purposes of determining a profile 
or profiles of the preservice competencies of Alberta 
education graduates. 

IV - Becoq}ing a Teacher 

This study is specific the University of Lethbridge and is 
primarily a research project. Nevertheless the results will have 
implications for the program and it fits well within the overall 
framework. The purpose of the study is to determine and examine 
the personal and program influences on the career development of 
teacher education candidates. A sample of 15 candidates who had 
just been admitted to the program were selected at random and 
agreed to be interviewed once each semester for the duration of 
the program and for three years after they leave the university. 
The methodology for this study is qualitative and emergent; it is 
also fascinating because of the richness of the data. I have 
gained the trust of these people and am able to obtain rich 
information about factors that affect their decisions, components 
of the profram that create stress, sources of support, how 
various program components affect their thinking and so on. This 
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study will add richness to the profiles of students developed 
from the more quaiititative studies on selection and developmert 
of candidates. 



V - Alternative Practicmn Experiences for Education Students 

A study which fits clearly within the second category of the 
overall framework is being conducted by a group of tiiose faculty 
members who, for pedagogical and practical reasons i decided to 
offer an alternative to the usual practicum experience for 
students in the first ''professional semester** of whe education 
program* A rationale was developed and a case made to the 
Faculty to offer two paired-placement experiences in one school, 
rather than the usual single placement in two different schools, 
for one group of students in Professional Semester I. A study was 
designed to collect data by means of surveys, interviews, logs 
and direct observation, on the effectiveness of the alternative 
proceduras and on teachers' and students' perceptions of the 
experience. The results of this study will then he u»ed in 
making decisions about practicum experiences for education 
students . 

IV - OTHER STUDIES 

The projrcts just discussed are those that clearly "fit" within 
the conceptual framework of program evaluation* However, to 
return to my spider web analogy there are strands which seem to 
escape the boundaries and to be only tangential ly a part of the 
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web. For exaaplei t^here are a number of studies underway which 
evaluate teaching effectiveness or examine some aspect of 
teaching, which have obvious implications for the education of 
teachers but which don't directly address our program. 
Similarly there are regular and frequent experi'-ents with 
different delivery systems, different aa*rangements of course 
offerings and so on. These are often evaluated only informally 
and decisions continue to be made on the basis of hunches and 
casual observations. We have attenpted to come to grips with 
some of those issues through the administrative framework for our 
prograjn evaluation proj ect . 

THE ADMINISTRATIVE FRAMEWORK 

In March of 1084 the Faculty of Education Council approved an 
administrative framework for enhancing the possibility that the 
results of evaluation studies will be used in making program 
decisions (see Figure 2) • The model outlines the procedures to 
be followed by the origins .^^rs of any study, whether formally 
designed or simply an ad-hoc experiment, which is likely to have 
implications for the teacher educ?tion program. The second 
aspect of the flow chart addresses the area of Faculty studies, 
for example studies in which data are collected from all students 
to address issues ox concern to the Faculty as a whole. 

It is entirely likely that the three studies currently underway 
on the selection, development aud competence of teacher education 
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candidates may result in decisions about data to be collected 
from all students i perhaps on admission to and exit from the 
program. However, given our experience with the QAULTEP project 
and with a large data base we felt it prudent this time to begin 
studying samples of stuuents before investing significant 
resources only to find we were addressing the wrong variables • 

It is too early to determine how effective the administrative 

framework will be. Projects are being registered with the 

Program Evaluation Committee and communication within the Faculty 

has improved « The real test will be in the dissemination and 

effect of the results of the various evaluation projects. 

CONCLUSION 

The University of Lethbridg«^ Faculty of Education is a small 
Faculty that likes to think big. When seen on paper a project/ 
such as the one described appears reasonable and manageable and 
even useful. We'd like to believe that it is all of those 
things. But there are still many problems. In spite of the fact 
that many faculty members were involved in the design, very few 
are involved in the implementation, simply because their research 
interests lie elsewhere. Committees are often inefficient and 
the administrative feedback mechanism has not been tested; indeed 
the mechanism may well prove ineffective as faculty members are 
not accustomed to reporting their research results to a 
committee. The human resources required to maintain a project 
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such as this are extensive. To date the momentum continues 
because of the commitoent of the Dean and one or two others who 
believe strongly in the importance of program evaluation; a 
change in personnel could chainge all of that. Nevertheless we 
believe we have ^^tarted something worthwhile. We believe that we 
can improve our teacher education program through this project. 
After all, if we weren't optimists we'd be out of business by 
now. 
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